Aesthetic Critiques Generation for Photos: Supplementary Material

As mentioned in our paper, we study a new problem, aesthetic critiques generation, which is different from conventional image captioning. Many recent works [1, 2] started to argued that conventional evaluation criteria (BLEU, METEOR and CIDEr) borrowed from machine translation community are unsuitable for image captioning task. How to choose a suitable criterion is still a tricky problem in image captioning, and this issue is more significant in our proposed new problem. In this section, we provide the results of our approaches on conventional automatic evaluation criteria. Then we give an example to explain why these criteria are improper. Compared to them, SPICE suits better to our task, although human evaluation could be regarded as a more reliable measure. The results of our approaches on BLEU, METEOR, and CIDEr are presented in Table 1. Compared to recent captioning works, the values shown in Table 1 are quite low. In the beginning, we are curious about why the automatic evaluation results are different from human’s (as shown in Section 5.2 of our main paper). So, we start to study whether these automatic criteria are reasonable. First, let us note that the ground truths of common image captioning and our PCCD datasets are different. In the former (e.g., MSCOCO and Flickr30k), an image is described by using multiple similar sentences, and thus the ground-truth captions are near-duplicates for the same image. In PCCD, an image is described by multiple sentences that are dissimilar (or not synonymous), and thus the ground-truth captions are